Skip to content

feat: FP8 requested changes#173

Merged
ani300 merged 1 commit intomainfrom
fp8-changes
Aug 6, 2025
Merged

feat: FP8 requested changes#173
ani300 merged 1 commit intomainfrom
fp8-changes

Conversation

@ani300
Copy link
Copy Markdown
Contributor

@ani300 ani300 commented Aug 6, 2025

Description of the change

  1. Pin torchao to 0.11 for deliverable
  2. Add an error check for torchao being installed when loading fp8 checkpoints
  3. Change scaling of Q and K transposed when using fp8 attention.

Related issues or PRs

Internal requests

How to verify the PR

Was the PR tested

  • I have added >=1 unit test(s) for every new method I have added (if that coverage is difficult, please briefly explain the reason)
  • I have ensured all unit tests pass

Checklist for passing CI/CD:

  • All commits are signed showing "Signed-off-by: Name <email@domain.com>" with git commit -signoff or equivalent
  • PR title and commit messages adhere to Conventional Commits
  • Contribution is formatted with tox -e fix
  • Contribution passes linting with tox -e lint
  • Contribution passes spellcheck with tox -e spellcheck
  • Contribution passes all unit tests with tox -e unit

Note: CI/CD performs unit tests on multiple versions of Python from a fresh install. There may be differences with your local environment and the test environment.

…ange how matrices are scaled in fp8 matmul

Signed-off-by: Antoni Viros i Martin <aviros@ibm.com>
Copy link
Copy Markdown
Collaborator

@tharapalanivel tharapalanivel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ani300 ani300 merged commit 28a2595 into main Aug 6, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants